Electronic device and control method for electronic device

ABSTRACT

An electronic device having a display screen that displays a user interface including at least one object, comprising: a display mechanism that displays the user interface; a coordinate detection mechanism that detects coordinates of a user touch point on the display screen; a first feedback presentation mechanism that presents tactile feedback at a prescribed point to make the user perceive the presence of an object; a measurement mechanism that measures a force generated by the user pressing the display screen; and a second feedback presentation mechanism that detects an operation trigger based on the force and generates mechanical vibration.

CROSS-REFERENCE TO RELATED APPLICATIONS

This Non-provisional application claims priority under 35 U.S.C. §119(a) on Patent Application No. 2018-085251 filed in Japan on Apr. 26,2018, and Patent Application No. 2018-221310 filed in Japan on Nov. 27,2018, the entire content of which is hereby incorporated by reference.

BACKGROUND

The present invention relates to an electronic device that providesfeedback on an object included in a user interface.

In recent years, electronic devices equipped with a touch panel such asa smartphone or a car navigation become popular.

When a user operates an object such as an icon included in a userinterface displayed via the touch panel, the electronic device activatesthe function corresponding to the object.

SUMMARY OF THE INVENTION

The user feels the same texture of the touch panel regardless of anypart of the panel, because the entire surface of the touch panel isuniformly hard. Thus, it is desirable to provide feedback to the user sothat the user can perceive the presence of an object, without activatingthe function corresponding to the object. Also, it is desirable toprovide feedback to a user so that the user can perceive that theoperation to activate the function has been accepted when the functioncorresponding to the object is activated.

As a technology to solve the above-mentioned problem, the technologydescribed in Japanese Patent Application Laid-open Publication No.2008-33739 is known. Japanese Patent Application Laid-open PublicationNo. 2008-33739 discloses (1) a device provides force feedback 302 sothat a user 2 can “feel” a GUI object 310 when the user 2 touches apoint that is in the hotspot or that intersects with the border of theGUI object 310 and when the pressure value is less than an activationthreshold value. (2) Thereafter, the user 2 presses the GUI object 310with a stronger force to increase the pressure value to a level greaterthan an activation threshold value 321. The GUI object 310 is activatedby this user's operation, and the device provides force feedback 304 andvisual feedback 305 in order to notify the user 2 that the GUI object310 is selected and activated.

An aspect of this disclosure is an optical element comprising: anelectronic device having a display screen that displays a user interfaceincluding at least one object, comprising: a display mechanism thatdisplays the user interface; a coordinate detection mechanism thatdetects coordinates of a user touch point on the display screen; a firstfeedback presentation mechanism that presents tactile feedback at aprescribed point to make the user perceive the presence of an object ameasurement mechanism that measures a force generated by the userpressing the display screen; and a second feedback presentationmechanism that detects an operation trigger based on the force andgenerates mechanical vibration.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a configuration example of anelectronic device of Embodiment 1;

FIG. 2A and FIG. 2B are diagrams for explaining the structure andoperation of an electrostatic tactile panel of Embodiment 1;

FIG. 3 is a flowchart for explaining the processes performed by theelectronic device of Embodiment 1;

FIG. 4A is a flowchart for explaining a touch detection processperformed by a press detection unit of Embodiment 1;

FIG. 4B is a flowchart for explaining a press detection processperformed by the press detection unit of Embodiment 1;

FIG. 5 is a diagram illustrating an example of a tactile presentation asa result of an operation performed on a display screen of the electronicdevice of Embodiment 1;

FIG. 6 is a diagram illustrating a misalignment between a displayposition of an object and a position of the object perceived throughfeedback in the electronic device of Embodiment 1;

FIG. 7A and FIG. 7B are diagrams illustrating a configuration example ofa tactile presentation device of Embodiment 2;

FIG. 8 is a flowchart for explaining baseline updating processesperformed by a control device of Embodiment 4; and

FIG. 9 is a diagram illustrating a misalignment between a displayposition of an object and a position of the object perceived throughfeedback on a conventional touch panel.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Below, embodiments of the present invention will be explained withreference to figures. However, the present invention shall not beinterpreted as limited to the descriptions of embodiments below. It iseasily understood by a person skilled in the art that the specificconfigurations of the present invention can be modified withoutdeparting from the concept and scope of the present invention.

Embodiment 1

FIG. 1 is a diagram illustrating a configuration example of anelectronic device 10 of Embodiment 1.

The electronic device 10 includes a tactile presentation device 100, acontrol device 110, and a storage device 120. The tactile presentationdevice 100 and the control device 110 are connected to each other via aconnection line, and the control device 110 and the storage device 120are connected to each other via a connection line.

The tactile presentation device 100 presents a user with a UT (userinterface) including at least one object, and accepts an operation viathe UT. In addition, the tactile presentation device 100 providesfeedback for the user to perceive the presence of an object included inthe UT, and feedback for the user to perceive that an operation on theobject has been accepted.

The tactile presentation device 100 includes an electrostatic tactilepanel/touch panel 101, a force sensor 102, a liquid crystal display(LCD) 103, a carrier 104, a base 105, a lateral actuator 106, and aplate spring 107. The respective components of the tactile presentationdevice 100 are stored in an appropriate ease. The electrostatic tactilepanel/touch panel 101, the force sensor 102, and the liquid crystaldisplay 103 are configured to realize a display screen for displayingthe UI, and the carrier 104, the lateral actuator 106, and the platespring 107 are configured to provide mechanical vibration.

The base 105 is a component serving as a base of the tactilepresentation device 100. The lateral actuator 106 and the plate spring107 are mounted on the base 105. Also, on the base 105, the carrier 104that vibrates due to the effects of the lateral actuator 106 and theplate spring 107 is mounted. The carrier 104 vibrates in a specificdirection with respect to the base 105. In the description below, thevibration of the carrier 104 generated by the lateral actuator 106 andthe plate spring 107 will also be referred to as mechanical vibration.

The lateral actuator 106 is a device that generates lateral movement inthe direction horizontal to the display device (such as X direction or Ydirection). The plate spring 107 is a mechanism to generate vibration inaccordance with the movement of the lateral actuator 106.

The carrier 104 is a component serving as a base to stack up therespective components of the display screen. On the carrier 104, theliquid crystal display 103, the force sensor 102, and the electrostatictactile panel/touch panel 101 are mounted.

In Embodiment 1, those components are configured to generate vibrationin a direction substantially parallel to the horizontal direction of thedisplay screen of the display device.

The liquid crystal display 103, the force sensor 102, and theelectrostatic tactile panel/touch panel 101 are placed so as to besubstantially parallel to the base 105.

The electrostatic tactile panel/touch panel 101 includes anelectrostatic tactile panel 200 (see FIG. 2) and a touch panel. Theelectrostatic tactile panel 200 provides a texture feeling of the objectusing the electrostatic force when the user drags the UI. The touchpanel is configured to detect the position of the finger of the user onthe display screen. The structure and operation of the electrostatictactile panel 200 will be explained with reference to FIGS. 2A and 2B.

The force sensor 102 measures the force applied by the finger of theuser to the display screen. In Embodiment 1, the force sensor 102measures the force in a direction substantially parallel to the verticaldirection of the display screen. That is, the force sensor 102 measuresthe force in the pressing direction. The liquid crystal display 103displays an image corresponding to the UI. The force sensor 102 outputsa difference (relative value) between the baseline, which is a referencevalue of the force (zero point), and the measurement value. In thedescription below, “force measurement” means to calculate the relativevalue, and the “measurement value of force” means the relative value.

The control device 110 is an arithmetic device that executes programs.Examples of the arithmetic device is a processor, GPU (GraphicsProcessing Unit), and FPGA (Field Programmable Gate Array). The storagedevice 120 stores therein programs and data used by the control device110. The storage device 120 is a memory, for example. The storage device120 includes a work area used by programs. The programs may be installedin the electronic device 10 in advance, or may be installed from astorage medium having the programs stored therein.

The control device 110 operates as a functional unit (module) to controlthe tactile presentation device 100. Specifically, the control device110 acts as a display control unit 111, a layout analyzing unit 112, anelectrostatic tactile control unit 113, a press detection unit 114, anda mechanical vibration tactile control unit 115.

The display control unit 111 controls the display of the UI on thedisplay screen Specifically, the display control unit 111 retrievessetting information of the UI from the storage device 120, and controlsthe liquid crystal display 103 according to the setting information sothat the UI including at least one object is displayed. The settinginformation of the UI includes the layout of the object, the attributeof the object, and the like. The attributes of the object includeinformation as to whether the feedback needs to be provided or not.

The layout analyzing unit 112 analyzes the setting information of theUI, and identifies the layout of the object and the attribute of theobject included in the UI. The layout analyzing unit 112 outputs thelayout of the object and the attribute of the object to theelectrostatic tactile control unit 113.

The electrostatic tactile control unit 113 controls the electrostatictactile panel/touch panel 101 so that the texture feeling is provided atthe position of a prescribed object on the display screen.

The press detection unit 114 detects the finger of the user touching thedisplay screen based on the output from the electrostatic tactilepanel/touch panel 101, and identifies the position of the finger of theuser on the display screen (touch position). The press detection unit114 detects a press event for the object based on the force measured bythe force sensor 102, the position of the user's finger on the displayscreen, and the setting information of the UI. The press event for theobject means an operation performed by the user to press down theobject. When detecting the press event for the object, the pressdetection unit 114 outputs a request to generate mechanical vibration tothe mechanical vibration tactile control unit 115 to make the userperceive that the operation performed on the object has been accepted.

The mechanical vibration tactile control unit 115 controls the lateralactuator 106 to generate mechanical vibration.

For each functional unit included in the control device 110, a pluralityof functional units may be integrated into one functional unit, or onefunctional unit may be divided into a plurality of functional units foreach function.

The electronic device 10 of Embodiment 1 includes a display mechanismthat displays the UI, a first feedback presentation mechanism thatpresents tactile feedback at a prescribed point to make the userperceive the presence of an object; a coordinate detection mechanismthat detects coordinates of a user touch point on the display screen; ameasurement mechanism that measures a force generated by the userpressing the display screen; and a second feedback presentationmechanism that generates mechanical vibration to perceive the user thatthe operation performed on the object has been accepted.

The configuration to realize the display mechanism is the liquid crystaldisplay (LCD) 103 and the display control unit 111, for example. Theconfiguration to realize the first feedback presentation mechanism isthe electrostatic tactile panel/touch panel 101 and the electrostatictactile control unit 113, for example. The configuration to realize thecoordinate detection mechanism is the electrostatic tactile panel/touchpanel 101 and the press detection unit 114, for example. Theconfiguration to realize the measurement mechanism is the force sensor102, for example. The configuration to realize the second feedbackpresentation mechanism is the carrier 104, the base 105, the lateralactuator 106, the plate spring 107, and the mechanical vibration tactilecontrol unit 115, for example.

FIG. 2A and FIG. 2B are diagrams for explaining the structure andoperation of the electrostatic tactile panel 200 of Embodiment 1. InEmbodiment 1, the electrostatic tactile panel 200 described in JapanesePatent Application Laid-Open Publication No. 2016-95832 is used for adevice to provide feedback.

The electrostatic tactile panel 200 is configured to a substrate onwhich a plurality of electrodes 210 and a plurality of electrodes 211are patterned, and a circuit (not illustrated in the figure) thatsupplies a voltage to each of the plurality of electrodes 210 and theplurality of electrodes 211. The substrate has an insulating layer 201and an electrode layer 202.

The electrodes 210 and the electrodes 211 are arranged in such a mannerthat the respective electrodes intersect with each other. Portions wherethe electrodes 210 and the electrodes 211 overlap intersect with eachother via an insulating film. Also, an insulating film is formed abovethe electrodes 210 and the electrodes 211. This way, when the usertouches the display screen, the finger and the respective electrodes 210and 211 are electrically insulated.

Below, the operation of the electrostatic tactile panel 200 to provide atexture feel to an object 220 displayed at the position as illustratedin FIG. 2A on the liquid crystal display 103 will be described.

The electrostatic tactile control unit 113 identifies the electrodes 210and 211 overlapping the object 220, and outputs, to the electrostatictactile panel 200, a control signal to apply a voltage to the identifiedelectrodes 210 and 211. The circuit of the electrostatic tactile panel200 outputs a voltage signal of a frequency required for the identifiedelectrodes 210 and 211 based on the control signal. As a result, atexture feel can be provided to the area where the object 220 isdisplayed, that is, a specific area of the display screen. In this way,since the texture feel can be presented in a prescribed area only, whena plurality of fingers are dragging the UI at the same time, a texturefeel is presented to the finger dragging the object 220 only, not to thefingers dragging anything other than the object 220. When a plurality ofusers are dragging the UI at the same time, a texture feel is presentedto the finger of the user who drags the object 220 only, not to thefingers of the user who drags anything other than the object 220.

The electrodes 210 and 211 not applied with a voltage are grounded sothat a voltage is not induced due to the capacitance coupling of theelectrodes. Instead of making them grounded, it is also possible toapply a DC voltage or a voltage of a frequency equal to or greater thana prescribed level.

The texture feel presented by applying a voltage to the electrodes 210and 211 overlapping the object 220 will be explained with reference toFIG. 2B.

When the user operates the electrostatic tactile panel/touch panel 101at the uppermost layer of the display screen, the friction force Fr isgenerated in the horizontal direction of the display screen, and theforce Fn is generated in the vertical direction of the display screen.

When the user performs an operation (dragging) of tracing the object 220with a finger, since the friction force on the object 220 differs fromthat of other areas, the user can feel the texture of the object 220. Inthe drag operation, the force Fn is sufficiently small.

When the user presses the object 220 on the electrostatic tactilepanel/touch panel 101 with a finger, the friction force Fr decreases andthe force Fn increases.

The plurality of electrodes 210 and the plurality of electrodes 211 maybe used to detect a touch made by an object such as a finger. In thisease, the plurality of electrodes 210 and the plurality of electrodes211 can be divided into a group of electrodes for presenting theelectrostatic tactile feedback, and a group of electrodes for detectinga contact of an object in a time divisional manner or space divisionalmanner. This makes it possible to realize the electrostatic tactilepanel/touch panel 101 that doubles as an electrostatic tactile panel anda touch panel. The pressure detection unit 114 detects a contact of thefinger based on the electrostatic capacitance generated between theelectrodes 210 and 211 due to an operation performed on the touch panel,and identifies the position of the contact.

The electrostatic tactile panel/touch panel 101 that doubles as anelectrostatic tactile panel and a touch panel can be realized byincorporating an optical touch sensor into an electrostatic tactilepanel.

Next, the process performed by the electronic device 10 will beexplained. FIG. 3 is a flowchart for explaining the processes performedby the electronic device 10 of Embodiment 1.

The display control unit 111 selects a UI to be displayed in the displayscreen based on the operation state and the like, and controls theliquid crystal display 103 based on the setting information of theselected UI, thereby updating the UI (Step S101).

Next, the layout analyzing unit 112 obtains the setting information ofthe selected UI from the storage device 120 (Step S102), and analyzesthe layout of the object in the UI (Step S103).

Next, the layout analyzing unit 112 selects electrostatic electrodepattern data to apply a voltage to the electrodes 210 and 211 of theelectrostatic tactile panel 200 based on the analysis results (StepS104). The electrostatic electrode pattern data specifies the size andfrequency of the voltage. The layout analyzing unit 112 outputs theanalysis results and the electrostatic electrode pattern data to theelectrostatic tactile control unit 113.

The electrostatic tactile control unit 113 presents electrostatictactile stimulus by applying a voltage to the predetermined electrodes210 and 211 of the electrostatic tactile panel 200 based on the analysisresults and the electrostatic electrode pattern data (Step S105).

Thereafter the press detection unit 114 starts a detection process (StepS106). In the detection process, a touch detection process for detectingthe contact of the user's finger on the electrostatic tactilepanel/touch panel 101 and a press detection process for detecting apress operation on the electrostatic tactile panel/touch panel 101 areexecuted.

If the display control unit 111 detects a need for switching the UIcaused by an operation of the user or the like, the process returns toStep S101.

FIG. 4A is a flowchart for explaining a touch detection processperformed by the press detection unit 114 of Embodiment 1.

The press detection unit 114 detects a signal indicating that the fingerof the user is in contact with the electrostatic tactile panel/touchpanel 101 (Step S201). The timing to detect the signal indicating thecontact of the user's finger can be set to any appropriate timing.

For example, the press detection unit 114 detects, as the signal, achange in electrostatic capacitance of the electrostatic tactilepanel/touch panel 101. A change in value of the force sensor 102 mayalso be detected as the signal.

Next, the press detection unit 114 calculates the coordinates indicatingthe touch position of the finger of the user on the electrostatictactile panel/touch panel 101 (Step S202). The coordinates arecalculated based on the position of the electrodes 210 and 211 in whicha change in electrostatic capacitance has been detected, for example.

Next, the press detection unit 114 stores the calculated coordinates inthe storage device 120 (Step S203). Thereafter, the press detection unit114 returns to Step S201.

The coordinates previously calculated may be deleted by overwriting itwith the newly calculated coordinates or may be managed as history data.

FIG. 4B is a flowchart for explaining a press detection processperformed by the press detection unit 114 of Embodiment 1.

The press detection unit 114 obtains the value of force measured by theforce sensor 102 (Step S211). The timing at which the force sensor 102measures the force may be set to any appropriate timing. The cycle toperform the touch detection process and the press detection process maybe the same or differ from each other.

The press detection unit 114 obtains the coordinates of the contactposition (pressed position) from the storage device 120.

The press detection unit 114 determines whether a press event for theobject has occurred or not based on the comparison results of the forcevalue and the threshold value and the coordinates of the contactposition (Step S212). For example, the press detection unit 114determines whether the force value is greater than the threshold valueor not and whether the coordinates of the contact position coincide withthe coordinates of the object or not. If both of the two conditions aremet, the press detection unit 114 determines that the press event forthe object has occurred. If the determining process used the force valueonly, the press event could erroneously be detected due to externalvibration and the like, which would cause unnecessary feedback to beprovided. In the present invention, the determining process uses both ofthe contact state and the force value, and as a result, the erroneousdetection of the press event can be prevented.

If the press detection unit 114 determines that the press event for theobject has not occurred, the press detection unit 114 returns to StepS211.

If the press detection unit 114 determines that the press event for theobject has occurred, the press detection unit 114 instructs themechanical vibration tactile control unit 115 to provide feedback viathe mechanical vibration (Step S213). Thereafter, the press detectionunit 114 returns to Step S211.

After receiving the instruction from the press detection unit 114, themechanical vibration tactile control unit 115 generates mechanicalvibration by controlling the lateral actuator 106. This causes thecarrier 104 to vibrate in a specific direction.

The touch detection process and the press detection process areperformed in parallel with each other as separate threads. Generally, adrag operation takes place more frequently than a press operation on theelectrostatic tactile panel/touch panel 101. If the touch detectionprocess and the press detection process were performed as the samethread, the touch detection process could not be performed until thepress detection process is completed. This would worsen the operationefficiency. By performing those two detection processes as separatethreads, however, the operation efficiency is improved.

FIG. 5 is a diagram illustrating an example of the tactile presentationas a result of the operation performed on the display screen of theelectronic device 10 of Embodiment 1.

As illustrated in FIG. 5, two objects 220 are displayed on theelectrostatic tactile panel/touch panel 101 at the uppermost layer ofthe display screen. The finger of the user transitions from the state(A) to the state (D) as illustrated.

The state (A) indicates that the user starts a drag operation of movingthe finger along the surface of the electrostatic tactile panel/touchpanel 101 to search for an object 220 displayed on the display screen.In this state, electrostatic tactile stimulus is presented at eachobject 220 by the electrostatic tactile control unit 113. Thus, when thefinger of the user passes through the object 220, the user can perceivea texture feel indicating the presence of the object 220.

The state (B) indicates that the user has completed the drag operationand the finger of the user has reached the prescribed object 220. Thestate (C) indicates that the object 220 is pressed by the user. In state(C), the finger is stopped, and the electrostatic tactile stimulus issmaller. In this case, the press detection unit 114 detects the pressevent for the object.

The state (D) indicates that mechanical vibrations are generated toperceive the user that the operation performed on the object 220 hasbeen accepted in response to the detection of the press event.

In the UI of the present invention, the user can perform a series ofoperations from searching for an object to pressing the object withoutmoving the finger away from the display screen. That is, seamless UI isprovided.

Next, the effects of the electronic device 10 described in Embodiment 1will be explained.

(Effect 1) The user can perceive the position of the object 220 byreceiving electrostatic tactile feedback. This means that the user canidentify position of the object 220 without relying on visualperception.

(Effect 2) The electrostatic tactile feedback for perceiving the user ofthe position of the object 220 is presented in a prescribed area aroundthe object 220 before the user starts performing an operation. FIG. 6 isa diagram illustrating a misalignment between the display position ofthe object 220 and the position of the object 220 perceived throughfeedback in the electronic device 10 of Embodiment 1. As illustrated inFIG. 6, there is no misalignment between the position of the object 220perceived through visual feedback and the position of the object 220perceived through tactile feedback in the electronic device 10 ofEmbodiment 1.

(Effect 3) The electronic device 10 is configured to presentelectrostatic tactile feedback to make the user perceive the presence ofthe object 220, and mechanical vibration tactile feedback to make theuser perceive the operation performed on the object 220. Those two typesof feedback are provided through different mechanisms, and it is easyfor the user to differentiate the two.

(Effect 4) The mechanical vibration acts in the direction substantiallyparallel to the horizontal direction of the display screen, andtherefore has a smaller effect on the measurement of the force in thevertical direction. Thus, it is possible to appropriately detect a pressevent for the object 220 while providing feedback to make the object 220perceived.

(Effect 5) In order to generate the mechanical vibration, largeelectricity is required. Thus, the conventional device had a problem ofrequiring large electricity to provide feedback. On the other hand,because the electronic device 10 of the present invention also uses theelectrostatic tactile feedback, it is possible to reduce the powerconsumption compared to the conventional device.

(Effect 6) In the conventional device, the mechanical vibration is usedfor both of the feedback to make the position of the object 220perceived and the feedback to perceive that the operation performed onthe object 220 has been accepted. However, the electronic device 10 ofthe present invention is configured to use the mechanical vibration forthe feedback to perceive that the operation performed on the object 220has been accepted only. This improves the life of the mechanism togenerate the mechanical vibration.

Embodiment 2

Embodiment 2 differs from Embodiment 1 in the configuration of thetactile presentation device 100. While the display type force sensor 102was used in Embodiment 1, a small-sized force sensor is used inEmbodiment 2.

FIG. 7A and FIG. 7B are diagrams illustrating a configuration example ofthe tactile presentation device 100 of Embodiment 2.

In FIGS. 7A and 7B, the liquid crystal display 103 is mounted on theelectrostatic tactile panel/touch panel 101.

On the base 105, the plate spring 107 is mounted so that the carrier 104vibrates in the direction substantially parallel to the horizontaldirection of the display screen. Also, on the base 105, the lateralactuator 106 is mounted to act on the carrier 104 in the directionsubstantially parallel to the horizontal direction of the displayscreen.

A beam 108 is mounted on the carrier 104 so as to form a cantilever insubstantially parallel with the horizontal direction of the displayscreen, and a strain gauge is mounted at a predetermined position of thebeam 108. The carrier 104 has a small-sized force sensor 102 constitutedof the beam 108 and the strain gauge. The beam 108 moves in the verticaldirection of the display screen as a result of the user pressing thedisplay screen.

On the frame on which the electrostatic tactile panel/touch panel 101 ismounted, columns 109 to affix the frame to the carrier 104 are disposed.

As illustrated in FIG. 7B, the carrier 104 vibrates in the directionsubstantially parallel to the horizontal direction of the display screenby the lateral actuator 106 and the plate spring 107.

The control method of the electronic device 10 of Embodiment 2 is thesame as the control method of the electronic device 10 of Embodiment 1,and therefore the description thereof is omitted.

By using the small-sized force sensor 102 for the tactile presentationdevice 100, instead of the display type force sensor 102, it is possibleto improve the display quality of the UI displayed on the displayscreen.

In the structure as illustrated in FIGS. 7A and 7B, the vibrationgenerated by the lateral actuator 106 is efficiently transmitted to theelectrostatic tactile panel/touch panel 101, which makes it possible togenerate large acceleration on the electrostatic tactile panel/touchpanel 101.

The features of the structure are described below. The lateral actuator106 is strongly coupled to the base 105, a linkage is strongly coupledto the lateral actuator 106, and the linkage and the carrier 104 arestrongly coupled with each other. Here, “strongly coupled” means thatthe components are coupled to each other without any play or loosenessat the joint in the vibration direction. In this embodiment, therespective components are fastened to each other strongly with screws.Furthermore, the carrier 104 and pedestals 130 are strongly coupled, thepedestals 130 and the beams 108 are strongly coupled, the beams 108 andthe columns 109 are strongly coupled, and the columns 109 and theelectrostatic tactile panel/touch panel 101 are strongly coupled.

As described above, from the lateral actuator 106 to the electrostatictactile panel/touch panel 101, the respective components are coupled toeach other without any play or looseness in the vibration direction, andtherefore, the vibration generated by the lateral actuator 106 istransmitted to the electrostatic tactile panel/touch panel 101efficiently without attenuated.

Embodiment 3

Embodiment 3 differs from Embodiment 1 in the method to provide feedbackto make the user perceive the position of the object. Specifically, theelectrostatic tactile panel 200 is replaced with another component.

One example is to replace the electrostatic tactile panel 200 with athermal display. The thermal display is configured to change the surfacetemperature. In order to provide feedback to make the user perceive theposition of an object, a display screen can be configured to have arraysof small-sized thermal displays, and by controlling the temperature ofthe thermal display corresponding to the display position of the objectso as to differ from that of other thermal displays, feedback toperceive the user of the object position can be provided.

Another example is to replace the electrostatic tactile panel 200 with adisplay made of magnetic fluid and arrays of electromagnet andconfigured to provide feedback to a prescribed position by actuating themagnetic fluid. By controlling the viscosity of the fluid in a positioncorresponding to the display position of the object, feedback toperceive the user of the object position can be provided.

Yet another example is to replace the electrostatic tactile panel 200with a display that has recesses and protrusions on the surface. Thisdisplay has a transparent polymer layer over a layer having very smallholes, and the recesses and protrusions on the surface are formed byinjecting fluid from those very small holes. By controlling the displayso that the fluid is injected through the very small holes correspondingto the display position of the object, feedback to perceive the user ofthe object position can be provided.

The control method for the electronic device 10 of Embodiment 3 is thesame as that of the electronic device 10 of Embodiment 1 except for thecontrol process to provide the feedback to make the presence of theobject 220 perceived, and therefore, the description thereof is omitted.

Embodiment 4

In Embodiment 4, the control device 110 updates the baseline. Becausethe size of the force applied to the force sensor 102 changes due toexternal environment such as temperature or atmospheric pressure, usagefrequency, and the like, it is necessary to update the baselineperiodically.

In order to update the baseline, the control device 110 needs toaccumulate a prescribed number of measurement values measured by theforce sensor 102 when no force is applied on the force sensor 102. Thus,the control device 110 needs to correctly identify the state in which noforce is applied to the force sensor 102, and accumulate the measurementvalues. If the baseline was updated using the measurement values thatwere measured when the force sensor 102 is applied with a force, itwould not be possible to appropriately detect the press event and thelike.

FIG. 8 is a flowchart for explaining baseline updating processesperformed by the control device 110 of Embodiment 4.

In this embodiment, the baseline updating process is performed by thepress detection unit 114. The control device 110 may alternativelyinclude a baseline updating unit that performs the baseline updatingprocess.

The cycle to perform the baseline updating process may be the same as ordiffer from the cycle to perform the detection process.

The press detection unit 114 obtains the value measured by the forcesensor 102 and stores the value in a buffer (Step S301). The buffer isincluded in the storage device 120.

The press detection unit 114 determines whether the electrostatictactile panel/touch panel 101 is pressed or not based on the measuredvalue of the force (Step S302). For example, the press detection unit114 determines whether the value of the force is greater than thethreshold value or not. The threshold value may be the same as or differfrom the threshold value used in Step S212.

If the press detection unit 114 determines that the electrostatictactile panel/touch panel 101 is pressed, the press detection unit 114clears the buffer (Step S104). Thereafter, the press detection unit 114returns to Step S301.

If the press detection unit 114 determines that the electrostatictactile panel/touch panel 101 is not pressed, the press detection unit114 determines whether a finger of the user is in contact with theelectrostatic tactile panel/touch panel 101 or not (Step S103). Forexample, the press detection unit 114 determines whether a signal isdetected or not. Alternatively, the press detection unit 114 may performthe touch detection process, and determines whether the finger is incontact or not based on the detection result.

If the press detection unit 114 determines that a finger of the user isin contact with the electrostatic tactile panel/touch panel 101, thepress detection unit 114 clears the buffer (Step S104). Thereafter, thepress detection unit 114 returns to Step S301.

If the press detection unit 114 determines that a finger of the user isnot in contact with the electrostatic tactile panel/touch panel 101, thepress detection unit 114 determines whether the number of measurementvalues accumulated is greater than a threshold value or not (Step S105).That is, the press detection unit 114 determines whether the number ofmeasurement values required to update the baseline has been reached ornot.

If the number of measurement values accumulated is equal to or smallerthan the threshold value, the press detection unit 114 returns to StepS301.

If the number of measurement values accumulated is greater than thethreshold value, the press detection unit 114 calculates a baselineusing the measurement values accumulated in the buffer, and sets thecalculated baseline (Step S306). Thereafter, the press detection unit114 returns to Step S301. For example, the press detection unit 114calculates the average value of the measurement values as the baseline.

Through the processes described above, the timing to update the baselinecan be appropriately controlled. As a result, it is possible toappropriately detect a press event and the like.

As set forth above, embodiments of this invention have been described;however, this invention is not limited to the foregoing embodiments.Although the embodiments have been described using a liquid crystaldisplay device as a representative case, the display device can be adifferent display device such as an organic EL display device. Thoseskilled in the art can easily modify, add, or convert each element inthe foregoing embodiment within the scope of this invention. A part ofthe configuration of one embodiment can be replaced with a configurationof another embodiment or a configuration of an embodiment can beincorporated into a configuration of another embodiment.

A part or the entirety of each of the above configurations, functions,processing units, processing means, and the like may be realized byhardware.

What is claimed is:
 1. An electronic device having a display screen thatdisplays a user interface including at least one object, comprising: adisplay mechanism that displays the user interface; a coordinatedetection mechanism that detects coordinates of a user touch point onthe display screen; a first feedback presentation mechanism thatpresents tactile feedback at a prescribed point to make the userperceive the presence of an object; a measurement mechanism thatmeasures a force generated by the user pressing the display screen; anda second feedback presentation mechanism that detects an operationtrigger based on the force and generates mechanical vibration.
 2. Theelectronic device according to claim 1, wherein the second feedbackpresentation mechanism is configured to: detect the force at theposition of the object as the operation trigger when the force isgreater than a threshold value; and generate the mechanical vibration tomake the user perceive that an operation performed on the object hasbeen accepted.
 3. The electronic device according to claim 2, whereinthe first feedback presentation mechanism is constituted of a substratehaving a plurality of electrodes patterned therein, and a circuit thatapplies a voltage to the plurality of electrodes, and wherein the firstfeedback presentation mechanism presents an electrostatic tactilefeedback by applying a voltage to at least one of the plurality ofelectrodes corresponding to a position of a prescribed object in thedisplay screen.
 4. The electronic device according to claim 2, whereinthe second feedback presentation mechanism generates mechanicalvibration in a direction substantially parallel to the direction that isperpendicular to a direction of the force.
 5. The electronic deviceaccording to claim 1, wherein the second feedback presentation mechanismand the display mechanism are connected through a connection mechanismthat reduces attenuation of the mechanical vibration.
 6. The electronicdevice according to claim 1, wherein the coordinate detection mechanismand the second feedback presentation mechanism operate independently ofeach other.
 7. The electronic device according to claim 1, furthercomprising an updating mechanism that updates a baseline used as areference value of the force, wherein the updating mechanism isconfigured to: store a force value measured by the measurement mechanismin a buffer; discard the force value stored in the buffer when thecoordinate detection mechanism detects the user touching the displaymechanism or when the measurement mechanism detects the user pressingthe display mechanism; and update the baseline based on the force valuesif the number of force values accumulated in the buffer is greater thana prescribed number.
 8. A control method for an electronic device havinga display screen that displays a user interface including at least oneobject, the electronic device including: a display mechanism thatdisplays the user interface; a coordinate detection mechanism thatdetects coordinates of a user touch point on the display screen; a firstfeedback presentation mechanism that presents tactile feedback at aprescribed point to make the user perceive the presence of an object; ameasurement mechanism that measures a force generated by the userpressing the display screen; and a second feedback presentationmechanism that detects an operation trigger based on the force and thatgenerates mechanical vibration, the control method for the electronicdevice including: a step of identifying, by the first feedbackpresentation mechanism, an object for which tactile feedback ispresented, based on setting information of the user interface; a step ofpresenting, by the first feedback presentation mechanism, tactilefeedback at a position of the identifies object in the display screen;and a step of detecting, by the second feedback presentation mechanism,the activation trigger when the force at the position of the identifiedobject is greater than a threshold value, and generating the mechanicalvibration to make the user perceive that an operation performed on theidentified object has been accepted.